Bounds for Frequency Estimation of Packet Streams

We consider the problem of approximating the frequency of frequently occurring elements in a stream of length n using only a memory of size m ≪ n. This models the process of gathering statistics on Internet packet streaming using a memory that is small relative to the number of classes (e.g. IP addresses) of packets. We show that when some data item a occurs αn times in a stream of length n, the FREQUENT algorithm of Demaine et al. [4], can approximate a’s frequency with an error of no more than (1−α)n/m. We also give a lower-bound of (1−α)n/(m+1) on the accuracy of any deterministic packet counting algorithm, which implies the FREQUENT algorithm is nearly optimal. Finally, we show that randomized algorithms can not be significantly more accurate since there is a lower bound of (1−α)Ω(n/m) on the expected accuracy of any randomized packet counting algorithm.